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Dear Sir: 

This paper is responsive to the Final Office Action dated August 8, 2007, to 
which a Notice of Appeal was filed on September 16, 2007. Reconsideration is 
respectfully requested. 

I. REAL PARTY IN INTEREST 

The entire interest in the present application has been assigned to Supersonic 
Aerospace International, LLC, a Nevada Limited Liability Company, having a place of 
business at 2250 E. Tropicana Avenue, Suite 1 9-121 , Las Vegas, NV 891 1 9, as 
recorded at reel 014299, frame 0743. 
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II. RELATED APPEALS AND INTERFERENCES 



An Appeal Brief for U.S. Patent Application No. 10/420,117 entitled "System and 
Method For Customizing Multiple Windows of Information On A Display" is being filed 
on the same day as the present Appeal Brief. While the scope of the claims differ, 
some of the claims in the present application and the '117 application include similar 
elements. 

ML STATUS OF CLAIMS 

Claims 1-14, 16-39, and 41-47 are pending in the application. Claims 1-14, 16- 
39 and 41-47 are rejected. 

Claims 1-6, and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Guell etal. (NPL Document "FLILO - an Enhanced Vision System", herein referred to 
as "Guell") in view of NASA et a/. (NPL Document "NASA's .High-Speed Research 
Program", herein referred to as "NASA") in further view of Snyder (US Patent No. 
6,381,519, herein referred to as "Snyder"). 

Claims 7-11, 13, and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Guell in view of NASA in further view of Snyder in further view of 
Bernier etal. (US Publication No. 2004/0169663, herein referred to as "Bernier"). 

Claims 16-19, 24-27, 33, 37, 38, and 46 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Guell in view of NASA in further view of Sankrithi et a/. (US 
Patent 6,405,975, herein referred to as "Sankrithi") in further view of Lipsanen et al. (US 
Publication No. 20020059614, herein referred to as "Lipsanen"). 

Claims 20-23, 28-32, 34-36, 39, 41-45, and 47 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Guell in view of NASA in further view of Sandrithi in further 
view of Lipsanen in further view of Bernier. 
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Claims 15 and 40 are canceled. 

The rejections of Claims 1-14, 16-39, and 41-47 are on appeal. 

IV. STATUS OF AMENDMENTS 

The appellant's response dated February 5, 2007, in response to the non-final 
office action dated October 17, 2006, was entered. 

V. SUMMARY OF CLAIMED SUBJECT MATTER 

Independent claim 1 pertains to a display system (50) (400) for a device 
comprising: 

computer executable instructions operable to: 

receive images from a first sensor and a second sensor representing 
scenery outside the device; (54, 56, 58, 60, 62) (FIG. 1A) 
(specification paragraphs [0024], [0035]-[0043]) 

detect moving objects in the images; (70) (FIG. 1A) (specification 
paragraphs [0026], [0045] ) 

fuse the images to a single viewpoint; (72) (FIG. 1A) (specification 
paragraphs [0046], [0047] ) 

transform the fused image to a first viewpoint image from a first operator 
station in the device and a second viewpoint image from a second 
operator station in the device, wherein the first and second viewpoint 
images conform to the scenery outside the device from each operator 
station; (76) (FIG. 1A) (specification paragraphs [0048], [0049]) 

generate a third display area associated with at least two mutually exclusive 
windows (1A, 1B) of information on a display device (100, 428) for the 
first operator station; (e.g., window 1C) (FIG. 4A) (specification 
paragraphs [0053]-[0055], [0057], [0058]) 

generate a third display area (100) associated with at least two mutually 
exclusive windows of information on another display device for the 
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second operator station; (FIG. 4A) (specification paragraphs [0053]- 
[0055], [0057], [0058]) 
wherein the third display areas area can be customized independently by the 
operators to display detailed information related to the information 
displayed in the associated windows, (e.g., associated windows 1A, 
1B; common window 1C) (FIG. 4A), (specification paragraphs 
[0053]-[0055], [0057], [0058]); (display 600, associated windows 
602, 604, common window 606) (FIG. 6), specification paragraphs 
[0064]-[0066]) 

Independent claim 16 pertains to a method for providing an out-the-window visual 
on a display device, comprising: 

receiving an image of a portion of the out-the-window visual scene from the 

viewpoint of a first type of sensor; (54, 56, 58, 60, 62) (FIG. 1A) (specification 
paragraphs [0024], [0035]-[0043]) 

receiving another image of a portion of the out-the-window visual scene from the 

viewpoint of another of the first type of sensor; (54, 56, 58, 60, 62) (FIG. 1A) 
(specification paragraphs [0024], [0035]-[0043]) 

fusing the images from the first type of sensors into a combined image to generate a 
first fused image; (72) (FIG. 1A) (specification paragraphs [0046], [0047] ) 

transforming the fused image to a first operator viewpoint and to a second operator 
viewpoint; (76) (FIG. 1 A) (specification paragraphs [0048], [0049]) and 

outputting the first operator viewpoint image to a first display device (100) (FIGS. 

1B, 4A) and the second operator viewpoint image to a second display device 
(100) (FIGS. 1B, 4A), wherein the display devices are positioned to provide 
the portion of a desired out-the window visual scene (102) (FIG. 1B) in 
combination with a window (104, 106) (FIG. 1B) that provides another 
portion of the desired out-the-window visual scene, and the viewpoint images 
are aligned with and scaled to conform to the out-the-window visual scene 
(76) (FIG. 1A). (specification paragraphs [0024]-[0033]). 
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Independent claim 24 pertains to a device, comprising: 

a display device; (100) (FIGS. 1B, 4A) and 

a display processor (52) (FIGS. 1B, 4A) operable to: 

receive a first sensor image representing a portion of scenery outside the 

device; (54, 56, 58, 60, 62) (FIG. 1A) (specification paragraphs [0024], 

[0035]-[0043]) 

transform the first sensor image to a viewpoint image from an operator station in 
the device, wherein the viewpoint image is sized and oriented to conform 
to the scenery outside the device from the operator station; (76) (FIGS. 
1A, 1B). (specification paragraphs [0024]-[0033]) and 

output the first operator viewpoint image to the display device, wherein the 
display device is positioned to provide the portion of a desired out-the 
window visual scene (102) (FIG. 1B) in combination with a window (104, 
106) (FIG. 1B) that provides another portion of the desired out-the-window 
visual scene, and the viewpoint image is aligned with and scaled to 
conform to the out-the-window visual scene (76) (FIG. 1A). (specification 
paragraphs [0024]-[0033]). 

Independent claim 37 pertains to an aircraft, comprising: 

a crewstation with cockpit windows (104, 106, 108); 

a first display device (100) for one crewmember; 

a second display device (100) for another crewmember; and 

a display processor (52) operable to: 

receive an image of an out-the-window visual scene from the viewpoint of 

a first type of sensor; (54, 56, 58, 60, 62) (FIG. 1 A) (specification 

paragraphs [0024], [0035]-[0043]) 
receive another image of a portion of the out-the-window visual scene 

from the viewpoint of another of the first type of sensor; (54, 56, 58, 

60, 62) (FIG. 1A) (specification paragraphs [0024], [0035]-[0043]) 
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fuse the images from the first type of sensors into a combined image to 

generate a first fused image; (72) (FIG. 1A) (specification 

paragraphs [0046], [0047]) 
transform the fused image to a first operator viewpoint and to a second 

operator viewpoint; (76) (FIG. 1A) (specification paragraphs [0048], 

[0049]) 

transform the first operator viewpoint image and the second operator 
viewpoint image to conform to the size and orientation of the out- 
the-window visual scene; (76) (FIGS. 1A, 1B). (specification 
paragraphs [0024]-[0033]) and 

output the first operator viewpoint image to the first display device (100) 
(FIGS. 1B, 4A) and the second operator viewpoint image to the 
second display device (100) (FIGS. 1B, 4A), wherein the display 
devices are positioned to provide the portion of a desired out-the 
window visual scene (102) (FIG. 1B) in combination with a cockpit 
window (104, 106, 108) (FIG. 1B) that provides another portion of 
the desired out-the-window visual scene, and the viewpoint images 
are aligned with and scaled to conform to the out-the-window 
visual scene. (76) (FIG. 1A) (specification paragraphs [0024]-[0033]). 

VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Whether independent claims 1,16, 24, and 37 and corresponding dependent 
claims 1-14 are unpatentable over Guell etal. (NPL Document "FLILO - an Enhanced 
Vision System", herein referred to as "Guell") in view of NASA et al. (NPL Document 
"NASA's High-Speed Research Program", herein referred to as "NASA") in further view of 
Snyder (US Patent No. 6,381,519, herein referred to as "Snyder"). 

Whether independent claims 16, 24, and 37 and corresponding dependent 
claims 17-23, 25-26, and 38, 39, 41-47 are unpatentable over Guell in view of NASA in 
further view of Sankrithi et al. (US Patent 6,405,975, herein referred to as "Sankrithi") in 
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further view of Lipsanen etal. (US Publication No. 20020059614, herein referred to as 
"Lipsanen"). 



VII. ARGUMENT 

Claims 1-6, and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Guell etal. (NPL Document "FLILO - an Enhanced Vision System", herein referred to 
as "Guell") in view of NASA et al. (NPL Document "NASA's .High-Speed Research 
Program", herein referred to as "NASA") in further view of Snyder (US Patent No. 
6,381,519, herein referred to as "Snyder"). Applicant respectfully disagrees. 
Independent claim 1 recites: 

"generate a third display area associated with at least two mutually 

exclusive windows of information on a display device for the first 
operator station; 

generate a third display area associated with at least two mutually 

exclusive windows of information on another display device for the 

second operator station; 
wherein the third display areas can be customized independently by the 

operators to display detailed information related to the information 

displayed in the associated windows." 

In contrast, the cited portions of Snyder do not teach or suggest a display device with a 
third display area associated with at least two mutually exclusive windows of 
information. In claim 1 , the two mutually exclusive windows of information and the 
third display area are on the same display device. In Snyder, the cited Figure 5 shows 
several mutually exclusive windows of information, but there is no third display area in 
Figure 5 that is associated with at least two of the mutually exclusive windows of 
information. All of the windows on the display in Figure 5 of Snyder are mutually 
exclusive. (Snyder col. 5 lines 8-18 teaches individual gages being replaced by a 
single CRT or LCD display 500 with graphical representations of an attitude indicator, 
airspeed indicator, altimeter, and horizontal situation indicator). Further, Snyder does 
not disclose or suggest allowing a crewmember to customize any area of display 500. 
Snyder only teaches crewmember interaction with display 304 and 306. (Snyder col. 5 
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line 29 through col. 6 line 55). Claim 1 is distinguishable from the cited references, 
alone and in combination, for at least these reasons. 

Claims 2-14 depend from claim 1 and include features that further distinguish 
them from the cited references. 

Independent claims 16, 24, and 37 and corresponding dependent claims 17-23, 
25-26, and 38, 39, 41-47 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Guell in view of NASA in further view of Sankrithi et al. (US Patent 6,405,975, herein 
referred to as "Sankrithi") in further view of Lipsanen et al. (US Publication No. 
20020059614, herein referred to as "Lipsanen"). Independent claims 16 and 37 recite: 
"the viewpoint images are aligned with and scaled to conform to the out-the-window visual 
scene." Independent claim 24 recites: "the viewpoint image is aligned with and scaled to 
conform to the out-the-window visual scene." These features are supported by at least 
paragraphs [0048] - [0049] in the specification. In contrast, Sankrithi teaches displaying 
images of the landing gear, but the landing gear images do not conform to nor are they 
aligned with the out-the-window visual scene. (Sankrithi col. 3 lines 44-52). Sankrithi 
does not teach or suggest displaying video images of anything other than the landing gear 
of the aircraft. (Sankrithi col. 3 line 54 - col. 8 line 22). 

Lipsanen is cited as teaching "the viewpoint images are aligned with and scaled to 
conform to the out-the-window visual scene", however if the images of the landing gear in 
Sankrithi were scaled to conform to the out-the-window visual scene, only a very small 
portion of the images of the landing gear would be visible on the crewstation displays. 
Applicant submits the combination of Sankrithi and Lipsanen would defeat the whole 
purpose of Sankrithi in aiding a crewmember in ground maneuvering large airplanes. 
Accordingly, there is no motivation or suggestion to combine Sankrithi and Lipsanen, nor 
would the combination of Sankrithi and Lipsanen have any reasonable expectation of 
success. 

Further, Lipsanen does not teach that the images are aligned with and scaled to 
conform to the out-the-window scene. Rather, Lipsanen teaches scaling, zooming, frame 
rate converting, and filtering as needed to display the video data on display 518. 

8 Of 19 10/615,634 

November 16, 2007 



(Lipsanen para. [0058]). Although Lipsanen mentions outputting video images to a flat 
panel display or a virtual display such as a head-mounted display, Lipsanen does not 
teach or suggest that the video images conform to the out-the-window scene. Id. 

Further still, the mobile multimedia terminals (MMT) of Lipsanen are for use by 
passengers, not crewmembers. (Lipsanen para. [0054], FIG. 5 shows diagram for a fixed 
seatback terminal 500a or passenger carried terminal 500b). The system in Lipsanen is 
used to distribute digital content to passengers in an airplane cabin. (Lipsanen, abstract). 
Thus, the suggested combination of references fails to teach all of the elements of claims 
16, 24, and 37. Additionally, there is no suggestion or motivation either in Sankrithi or 
Lipsanen to scale the video images to conform to an out-the-window scene. 

Accordingly, independent claims 16, 24, and 37 are distinguishable from the cited 
references, alone and in combination, for at least these reasons. 

Claims 17-23, 25-36, and 38, 39, 41-47 depend from Claims 16, 24, and 36, 
respectively, and include features that further distinguish them from the cited references. 

Allowance of claims 1-14, 16-39, and 41-47 is respectfully requested. 



I hereby certify that this correspondence is being transmitted to 
the USPTO on the date shown below: 



/Mary Jo Bertani/ 



(Signature) 
Mary Jo Bertani 



(Printed Name of Person Signing Certificate) 
November 16, 2007 



(Date) 



Respectfully submitted, 
/Mary Jo Bertani/ 

Mary Jo Bertani 
Attorney for Applicant(s) 
Reg. No. 42,321 
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VIM. CLAIMS APPENDIX 



Claims remaining in the application are as follows: 

1 . (Previously presented) A display system for a device, comprising: 
computer executable instructions operable to: 

receive images from a first sensor and a second sensor representing 

scenery outside the device; 
detect moving objects in the images; 
fuse the images to a single viewpoint; 

transform the fused image to a first viewpoint image from a first operator 
station in the device and a second viewpoint image from a second 
operator station in the device, wherein the first and second 
viewpoint images conform to the scenery outside the device from 
each operator station; and 

generate a third display area associated with at least two mutually 

exclusive windows of information on a display device for the first 
operator station; 

generate a third display area associated with at least two mutually 

exclusive windows of information on another display device for the 

second operator station; 
wherein the third display areas can be customized independently by the 

operators to display detailed information related to the information 

displayed in the associated windows. 

2. (Original) The display system of Claim 1, further comprising: 
computer executable instructions operable to: 

combine the first and second viewpoint images with symbols, wherein the 
symbols represent information regarding the operational state of 
the device and the moving objects detected in the images. 
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3. (Original) The display system of Claim 1 , wherein the instructions for 
detecting moving objects in the first sensor image are configured to execute in a first 
processor, and the instructions for detecting moving objects in the second sensor 
image are configured to execute in a second processor simultaneously with the 
instructions in the first processor. 

4. (Original) The display system of Claim 3, wherein the instructions for 
transforming the fused image to the first viewpoint image are configured to execute in 
the first processor, and the instructions for transforming the fused image to the second 
viewpoint image are configured to execute in the second processor. 

5. (Original) The display system of Claim 2, wherein the symbols represent the 
moving objects in the vicinity of the device. 

6. (Original) The display system of Claim 2, wherein at least one of the first and 
second viewpoint images include environmental information for the area where the 
device is operating. 

7. (Original) The display system of Claim 2, wherein the symbols represent 
weather hazards in the vicinity of the device. 

8. (Original) The display system of Claim 2, wherein the computer executable 
instructions are further operable to receive an enhanced image from a third sensor 
configured to provide an image of the out-the-window scenery in low-visibility 
conditions 

9. (Original) The display system of Claim 8, wherein the computer executable 
instructions are further operable to fuse the single viewpoint image with the enhanced 
image. 
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10. (Original) The display system of Claim 9, wherein the computer executable 
instructions are further operable to utilize data from at least one position sensor to 
determine the location of the objects with respect to the device. 

1 1 . (Original) The display system of Claim 2, wherein the computer executable 
instructions are further operable to utilize data from off-board data sources regarding 
the objects. 

12. (Original) The display system of Claim 1 , wherein the first sensor and the 
second sensor are video cameras. 

13. (Original) The display system of Claim 8, wherein the third sensor is a 
RADAR sensor. 

14. (Original) The display system of Claim 8, wherein the third sensor is a FLIR 
sensor. 

15. (Canceled) 

16. (Previously presented) A method for providing an out-the-window visual 
scene on a display device, comprising: 

receiving an image of a portion of the out-the-window visual scene from the 

viewpoint of a first type of sensor; 
receiving another image of a portion of the out-the-window visual scene from the 

viewpoint of another of the first type of sensor; 
fusing the images from the first type of sensors into a combined image to 

generate a first fused image; 
transforming the fused image to a first operator viewpoint and to a second 

operator viewpoint; and 
outputting the first operator viewpoint image to a first display device and the 

second operator viewpoint image to a second display device, wherein the 

display devices are positioned to provide the portion of a desired out-the 
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window visual scene in combination with a window that provides another 
portion of the desired out-the-window visual scene, and the viewpoint 
images are aligned with and scaled to conform to the out-the-window 
visual scene. 

17. (Original) The method of Claim 16, further comprising detecting objects in 
the first fused image from the first type of sensor. 

18. (Original) The method of Claim 1 7, further comprising combining the first 
fused image with symbols representing the objects. 

1 9. (Original) The method of Claim 1 6, further comprising transforming the first 
operator viewpoint image and the second operator viewpoint image to conform to the 
out-the-window visual scene. 

20. (Original) The method of Claim 1 6, further comprising fusing the first fused 
image with an enhanced image of a portion of the out-the-window scenery from at least 
one of the group of a RADAR sensor and a FLIR sensor, to generate a second fused 
image. 

21 . (Original) The method of Claim 1 6, further comprising fusing the second 
fused image with an enhanced image of a portion of the out-the-window scenery from 
at least one of the group of a RADAR sensor and a FLIR sensor, to generate a second 
fused image. 

22. (Original) The method of Claim 21, further comprising: transforming the 
second fused image to the first operator viewpoint and to the second operator 
viewpoint. 

23. (Original) The method of Claim 20, further comprising: providing portions of 
the transformed image with data from a terrain map database. 
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24. (Previously presented) A device, comprising: 
a display device; and 

a display processor operable to: 

receive a first sensor image representing a portion of scenery outside the 
device; 

transform the first sensor image to a viewpoint image from an operator 
station in the device, wherein the viewpoint image is sized and 
oriented to conform to the scenery outside the device from the 
operator station; and 
output the first operator viewpoint image to the display device, wherein the 
display device is positioned to provide the portion of a desired out-the 
window visual scene in combination with a window that provides another 
portion of the desired out-the-window visual scene, and the viewpoint 
image is aligned with and scaled to conform to the out-the-window visual 
scene. 

25. (Original) The device of Claim 24, wherein the display processor is further 
operable to combine the viewpoint image with symbols, wherein the symbols represent 
information regarding the operational state of the device and the moving objects 
detected in the images. 

26. (Original) The device of Claim 24, wherein the display processor is further 
operable to detect moving objects in the first sensor image 

27. (Original) The device of Claim 24, wherein the display processor is further 
operable to generate symbols representing moving objects in the sensor image and the 
operational state of the device. 

28. (Original) The device of Claim 24, wherein the display processor is further 
operable to generate symbols representing weather hazards in the vicinity of the 
device. 
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29. (Original) The device of Claim 24, wherein the display processor is further 
operable to receive an enhanced image of the out-the-window scenery in low-visibility 
conditions from a second sensor. 

30. (Original) The device of Claim 29, wherein the display processor is further 
operable to fuse the viewpoint image with the enhanced image. 

31 . (Original) The device of Claim 26, wherein the display processor is further 
operable to utilize data from at least one position sensor to determine the location of 
the objects with respect to the device. 

32. (Original) The device of Claim 26, wherein the display processor is further 
operable to utilize data from off-board data sources regarding the objects. 

33. (Original) The device of Claim 24, wherein the sensor is a video camera. 

34. (Original) The device of Claim 29, wherein the second sensor is a RADAR 
sensor. 

35. (Original) The device of Claim 29, wherein the second sensor is a FLIR 
sensor. 

36. (Original) The device of Claim 24, wherein the display processor is further 
operable to generate a common display area associated with at least two mutually 
exclusive windows of information on the display device, wherein the common display 
area can be customized by the operator to display detailed information related to the 
information displayed in the associated windows. 

37. (Previously presented) An aircraft, comprising: 
a crewstation with cockpit windows; 

a first display device for one crewmember; 

a second display device for another crewmember; and 
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a display processor operable to: 

receive an image of an out-the-window visual scene from the viewpoint of 

a first type of sensor; 
receive another image of a portion of the out-the-window visual scene 

from the viewpoint of another of the first type of sensor; 
fuse the images from the first type of sensors into a combined image to 

generate a first fused image; 
transform the fused image to a first operator viewpoint and to a second 

operator viewpoint; 
transform the first operator viewpoint image and the second operator 
viewpoint image to conform to the size and orientation of the out- 
the-window visual scene; and 
output the first operator viewpoint image to the first display device and the 

second operator viewpoint image to the second display device, wherein 
the display devices are positioned to provide the portion of a desired out- 
the window visual scene in combination with a cockpit window that 
provides another portion of the desired out-the-window visual scene, and 
the viewpoint images are aligned with and scaled to conform to the out- 
the-window visual scene. 

38. (Original) The aircraft of Claim 37, wherein the display processor is further 
operable to detect objects in the first fused image from the first type of sensor. 

39. (Original) The aircraft of Claim 38, wherein the display processor is further 
operable to combine the first fused image with symbols representing the objects and 
primary flight information for the aircraft. 

40. (Canceled) 

41 . (Original) The aircraft of Claim 37, wherein the display processor is further 
operable to fuse the first fused image with an enhanced image of a portion of the out- 
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the-window scenery from at least one of the group of a RADAR sensor and a FLIR 
sensor, to generate a second fused image. 

42. (Original) The aircraft of Claim 41 , wherein the display processor is further 
operable to fuse the second fused image with an enhanced image of a portion of the 
out-the-window scenery from at least one of the group of a RADAR sensor and a FLIR 
sensor, to generate a second fused image. 

43. (Original) The aircraft of Claim 42, wherein the display processor is further 
operable to transform the second fused image to the first operator viewpoint and to the 
second operator viewpoint. 

44. (Original) The aircraft of Claim 43, wherein the display processor is further 
operable to provide portions of the transformed images using data from a terrain map 
database. 

45. (Original) The aircraft according to Claim 37 further comprising: 

a terrain database coupled to provide the display processor with at least a portion of 
the required out-the-window field of view on the display device. 

46. (Original) The aircraft of Claim 37, wherein the display processor is further 
operable to display one of the operator viewpoint displays to the operator acting as pilot 
in command of the aircraft during a predefined aircraft operational state, and to allow 
the pilot in command to choose an option on the display device to view detailed 
information about the aircraft and aircraft subsystems during other aircraft operational 
states. 

47. (Original) The aircraft of Claim 46, wherein the display processor is further 
operable to generate a common display area associated with at least two mutually 
exclusive windows of information on each display device, wherein the common display 
area can be customized by the operator to display detailed information related to the 
information displayed in the associated windows. 



17 of 19 10/615,634 

November 16, 2007 



IX. EVIDENCE APPENDIX 



None. 
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X. RELATED PROCEEDINGS APPENDIX 



None. 
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